Spoken language is the most natural and convenient communication tool for people. Advances in speech and language technology have allowed an increased use of spoken language interfaces with a variety of different machines and computer systems. Interfaces to various systems and services through natural voice requests offer people convenience and efficiency in accessing variety of information from internet selectively. This is especially important for applications in eye-busy and hand-busy situations, such as driving a car or performing sophisticated computing tasks. Human machine interfaces that utilize spoken commands and voice recognition are generally based on dialog systems. A dialog system is a computer system that is designed to converse with a human using a coherent structure and text, speech, graphics, or other modalities of communication on both the input and output channel. Dialog systems that employ speech are referred to as spoken dialog systems and generally represent the most natural type of human machine interface. With the ever-greater reliance on electronic devices, spoken dialog systems are increasingly being implemented in many different systems.
In many human-machine interaction (HMI) systems, users can interact with the system through multiple input devices or types of devices, such as through voice input, gesture control, and traditional keyboard/mouse/pen inputs. This provides user flexibility with regard to data input and allows users to provide information to the system more efficiently and in accordance with their own preferences.
Present HMI systems typically limit particular modalities of input to certain types of data, or allow the user to only use one of multiple modalities at one time. For example, a vehicle navigation system may include both a voice recognition system for spoken commands and a touch screen. However, the touch screen is usually limited to allowing the user to select certain menu items by contact, rather than through voice commands. Such multi-modal systems do not coordinate user commands through the different input modalities, nor do they utilize input data for one modality to inform and/or modify data for another modality. Thus, present multi-modal systems do not adequately provide a seamless user interface system in which data from all possible input modalities can be used to provide accurate information to the system.
One common example of an HMI is the interface that a motor vehicle presents to a user and other occupants in the vehicle. Modern motor vehicles often include one or more in-vehicle information systems that provide a wide variety of information and entertainment options, which are referred to as “infotainment” in this document, to occupants in the vehicle. Common services that are provided by the in-vehicle infotainment systems include, but are not limited to, vehicle state and diagnostic information, navigation applications, hands-free telephony, radio and music playback, and traffic condition alerts. In-vehicle infotainment systems often include multiple input and output devices. For example, traditional buttons and control knobs that are used to operate radios and audio systems are commonly used in vehicle infotainment systems. More recent forms of vehicle input include touchscreen input devices that combine input and display into a single screen, as well as voice-activated functions where the in-vehicle infotainment system responds to voice commands. Examples of output systems include mechanical instrument gauges, output display panels, such as liquid crystal display (LCD) panels, and audio output devices that produce synthesized speech. The system state information, also called vehicle context information, can be useful to decide when, what, and how to select and/or present internet information to the driver or vehicle occupants.
In-vehicle infotainment systems operate using either minimally interactive or fully interactive information services. Minimally interactive services include many traditional systems such as in-vehicle radios and music playback devices. In a minimally interactive service, the user has a small degree of input over the content, such as selecting a radio station or changing audio tracks for pre-recorded music playback. The user has little or no control of the content in a minimally interactive service. Fully interactive services give the user much more control over the selection of information. Examples of fully interactive services include, for example, web browsers, interactive chat and communication applications, social networking applications, news, sports, and the like. Fully interactive services provide the user with a greater degree of flexibility, but many fully interactive services are unsuitable for use while the vehicle is in motion. The fully interactive services tend to be more complex than minimally interactive services, and a user cannot use the full-interactive services effectively while operating the vehicle. Consequently, improvements to HMIs and digital information systems that provide more flexibility to the user than the minimally interactive systems while also reducing the complexity of fully interactive systems would be beneficial.